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Work  in  this,  and  in  other  laboratories , denxanstratcd  that  cognitive 
events  can  be  inferred  from  cortical  potentials  which  can  be  recorded 
through  the  scalp  of  intact,  awake  human  subjects . Specific  components  of 
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surprising  event,  or  the  processing  of  tas/:  relevant  information . f-/e 
’jave  developed  the  use  of  Princi pal  Component  Analysis  for  the  dissociation 
of  the  ERP  components.  Using  this  technique  we  have  been  able  to  greatly 
refine  the  resolution  with  which  we  can  dissect  the  ERP  into  cognitively 
relevant  components.  At  the  same  time  wc  have  shown  that  Stepwise  Dis- 
criminant Analysis  can  be  used  to  detect  and  identify  the  brain  responses 
without  requiring  the  averaging  of  multiple  responses . 

It  has  become  quite  evident  that  a useful  lexicon  of  evoked  response 
components  exists  and  that  "signal  trial  analysis  techniques"  can  be 
developed . It  appeared  advisable  therefore  to  terminate  this  three  year 
project  at  the  end  of  its  second  year  and  to  initiate  a five  year  project 
which  would  include  as  a major  component  the  development  and  testing  of 
practical  applications  of  biocybernetic  communication  channels . A 
description  of  the  new  plan  is  included  in  this  report.  An  annotated 
bibliography  of  all  reports  developed  under  this  contract  follows. 
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INTRODUCTION 


The  present  document  constitutes  the  Final  Report  on  the  effort 
conducted  in  this  laboratory  under  contract  ftDAHC-15-7 3-C-0J18  bet  een 
July  1,  1973  and  June  30,  19,5.  We  originally  contracted  to  perform 
during  a three  year  period  a series  of  investigations  intended  to  assess 
the  feasibility  of  using  electrocortical  activity  monitor  human  decision 
making  within  the  context  of  ARPAs  Biocybernetic  Program.  The  prime  intent 
of  the  project  was  to  determine  the  extent  to  which  a lexicon  of  evoked 
brain  response  components  can  be  developed . Such  a lexicon  should  allow 
strong  inferences  about  decision  making  activities  by  the  human  operator  to 
be  based  on  electrocortical  recordings . We  were  also  to  develop  procedures 
for  detecting  and  identifying  the  brain  responses  by  techniques  which  do  not 
require  the  averaging  of  multiple  signals.  These  "single  trial"  techniques 
were  deemed  crucial  for  any  future  applications  of  the  evoked  response 
vocabulary  in  enhancing  the  performance  of  man-machine  systems. 

During  the  second  year  of  the  project  it  became  quite  evident  that  a 
useful  lexicon  of  evoked  response  components  exists  and  that  "signal  trial 
analysis  techniques"  can  be  developed . In  particular  the  stepwise  discriminant 
analysis  approach  we  have  been  evaluating  provide  a promising  tool  in  the 
development  of  real-life  applications  of  biocybernetic  communication 
channels . It  appeared  advisable , therefore , to  propose  the  termination  of 
this  three  year  project  at  the  end  of  its  second  year  and  to  initiate  a five 
year  project  wiich  would  include  as  a major  component  the  development  and 
testing  of  practical  applications  of  biocybernetic  communication  channels . 

A proposal  to  this  effect  was  submitted  to  ARPA  on  Aar. l!  4 1975  and  has 

been  approved . Thus  on  July  1,  1975  this  contract  has  been  terminated  and 
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the  work  in  the  laboratory  is  now  continuing  under  contract  IfUS  NAVY 
N-000-14-7 6-C-0002  monitored  by  the  Office  of  Naval  Research.  As  the 
project  has  not  in  fact  been  terminated  and  the  work  is  continuing  albeit 
under  a new  organizational  arrangement  an  effective  way  to  report  and 
summarize  the  achievements  of  the  past  two  years  is  to  present : the  bulk  of 
the  material  included  in  the  proposal  which  summarized  the  original  project 
and  outlined  the  plans  for  the  continuation  of  the  effort.  This  document 
consists  of  material  included  in  the  Research  Proposal  entitled  "The 
Vocabulary  of  Drain  Potentials : Inferring  Cognitive  Events  from  Brain 

Potentials  in  Operational  Settings,"  submitted  to  ARPA  by  the  University 
of  Illinois  on  April  4,  1975.  We  have  excluded  passages  pertaining  to  the 
management  and  budget  of  the  continuing  operation . This  report  also  includes 
a detailed , annotated,  bibliography  of  reports ■ on  work  supported  by  this 
contract  and  published  in  the  scientific  literature.  Material  is  thus 
available  for  several  classes  of  readers.  Those  interested  in  a summary  of 
the  effort  will  find  it  in  the  initial,  narrative,  section.  Those  interested 
in  a detailed  review  of  our  procedures  and  findings  can  find  this  through  the 
annotated  bibliography , and  finally,  a detailed  summary  of  our  projected 
experimental  plans  are  presented  in  the  final  sections  of  the  report. 


NARRATIVE  SUMMARY 


1 . Rationale 

We  have  demonstrated  that  cognitive  events  can  he  inferred  from  cortical 
potentials  which  can  be  recorded  through  the  scalp  of  intact , awake  human 
subjects . Specific  components  of  these  event  related  potentials  (ERPs)  have 
been  shown  to  be  manifestations  of  such  cognitive  events  as  the  preparation 
to  perform  a response,  the  preparation  to  intake  and  process  information,  the 
registration  of  a surprising  event,  or  the  processing  of  task  relevant  infor- 
mation. With  the  application  of  powerful  analytical  techniques , ^ we 

have  in  the  past  few  months  been  able  to  greatly  refine  the  resolution  with 
which  we  can  dissect  the  ERP  into  cognitively  relevant  components . ^ We 
will  refer  to  this  phase  of  the  work  as  the  determination  of  the  ERP  vocabu- 
lary. 

A brief  suimary  of  the  available  data  on  ERP  component  which  are  demon- 
strably associated  with  cognitive  events  follows. 

An  ERP  is  a transient  series  of  voltage  fluctuations  induced  in  cortical 
tissue  immediately  following  the  occurrence  of  some  critical  inducing  event. 
The  ERPs  can  be  induced  by  a variety  of  events  such  as  the  presentation  of 
a stimulus , the  occurrence  of  a response , a change  in  the  rate  of  stimulation 
etc.  The  amplitude  of  the  response , as  recorded  from  the  scalp,  is  minute 
relative  to  the  on-going  brain  wave  activity  (EEG) , and  it  is  only  through 
the  use  of  signal-averaging  techniques  that  the  waveform  and  characteristics 
of  ERPs  can  be  studied. 

An  ERP  consists  of  a series  of  positive  and  negative  peaks  which  continu 
for  about  500  msec.  The  waveform,  i.e.,  the  sequence  and  timing  of  positive 
and  negative  peaks,  depends  on  the  eliciting  event,  on  the  state  of  the  sub- 
ject, and  on  the  placement  of  the  recording  electrodes  on  the  scalp.  For  any 
given  set  of  conditions  ERPs  are  quite  consistent  between  individuals , yet 
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eac:h  individual  tends  to  display  a characteristic  wave  pattern.  In  the  few 
published  reports  of  ERPs  recorded  from  the  same  individual  over  long  periods 
the  wave  patterns  proved  remarkably  stable. 

Each  peak-to-trough,  or  base-to-peak  deflection  appearing  in  the  ERP 
at  a consistent  temporal  interval  following  the  eliciting  event  is  called  a 
" component Many  components  are  " exogenous ,"  that  is  they  represent  the 
response  of  cortical  tissue  to  the  arrival  of  sensory  volleys  as  a result  of 
the  activation  of  a peripheral  sense  organ.  The  exogenous  components  are 
often  followed  by  what  we  shall  call  endogenous  components.  These  represent 
cortical  inform tion  processing  invoked  by  the  psychological  demands  of  the 
context  in  which  a stimulus  is  presented  rather  than  by  the  physical  proper- 
ties of  the  stimulus.  For  example,  we,  and  others,  have  shown  that  if  a 
stimulus  provides  task-relevant  information  to  a subject  the  ERP  elicited 
by  that  stimulus  will  be  characterized  by  a large  positive  going  wave  with 
a latency  to  the  peak  of  about  300  msec.  We  call  this  component  P300 . 

With  few  exceptions,  ERP  components  are  labeled  by  a <character><number> 
designation.  The  character  indicates  the  polarity  of  the  component  (P  = posi- 
tive, N « negative),  the  number  refers  to  the  modal  delay  between  the  eliciting 
event  and  the  peak  of  the  component. 

1 * £i££-  Present  in  ERPs  elicited  by  all  stimuli  of  moderate  and  high 

intensities.  Recent  evidence  suggests  that  the  amplitude  of  N100 
is  a function  of  the  degree  to  which  subject  is  attending  selective- 
ly to  the  modality  in  which  the  stimuli  are  presented . 

2.  N190.  This  ERP  component , studied  in  detail  in  our  laboratory , seems 

to  he  elicited  whenever  a rare,  or  unexpected,  event  occurs.  It  is 
of  particular  interest  because  it  can  be  elicited  by  stimuli  which 
are  in  fact  m the  periphery  of  the  subject's  attention  span.  Due 
to  the  fairly  recent  discovery  of  N190  the  details  of  its  relationship 
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to  psychologies!  variables  have  yet  to  he  worked  out. 

This  is  the  most  robust  of  the  endogenous  ERP  components . 

In  more  than  a decade  of  research  it  has  been  recorded  in  many 
laboratories , in  a large  number  of  situations . It  seems  to  be 
most  reliably  recorded  in  association  with  task-relevant , rare 
stimuli  and  seems  to  reflect  in  amplitude  the  complexity  of  the 
information  processing  invoked  by  the  stimulus . 

4 * --SJ9.wJVave"  • have  established  that  the  P300  component  is  often 
followed  by  a slow  potential  shift  which  is  affected  by  the  same 
variables  which  are  known  to  affect  the  P300  except  that  it  has  a 
different  scalp  distribution.  Whereas  P300  appt'ars  largely  as  a 
P°sitive  going  pxotential  peaking  on  the  pari etal  (posterior)  scalp, 
the  slow  wave  is  positive  going  on  parietal  electrodes  and  negative 
going  in  frontal  electrodes. 

5’  Contingent  Negative  Variation  (The  CNV) . This  wave  precedes  anti- 

cipated events  in  response  to  which  the  subject  will  have  to  perform 
a demanding  motor,  or  mental,  act.  It  is  very  reliably  recorded 
during  the  warning  interval  (foreperiod)  in  a reaction  time  task, 
in  the  form  of  a negative  ramp  beginning  about  400  msec  after  the 
warning  stimulus  and  peaking  just  prior  to  the  response-commanding 
(imperative)  stimulus. 

6.  Readiness  Potential"  (RP) . This  is  a CNV-like  wave.  It  is  distinct 
only  in  the  sense  that  it  appears  prior  to  self-paced  voluntary  re- 
sponses- Its  occurrence  i.c  independent  of  the  presence  of  an  elici- 
ting, or  a command  stimulus . The  relationship  between  the  RP  and  the 
CN\,  is  a subject  of  much  active  research  and  some  controversy . 

The  determination  of  the  ERP  vocabulary  consists  of  defining  with  eve- 
increasing  resolution  the  specific  psychological  variables  which  determine 


6- 


t-'O  'mplitude  and  scalp  distribution  of  these  potentials . It  is  also 
i'.  < . ary  to  develop  techniques  for  dissecting  any  giv^n  waveform  into  its 

constituent  components . These  have  been  the  main  charges  of  our  past  re- 
search and  we  feel  we  made  substantial  progress . Specifically , we  have 
developed  a powerful  combination  of  Principal  Component  analysis  and  analysis 

of  variance  which  has  enabled  us  to  clarify  the  relationship  between  ERP 

* (4) 

components . 

As  the  known  vocabulary  of  ERPs  develops  it  becomes  increasingly  likely 
that  the  ERP  might  play  an  important  role  in  man-computer  interactions . If 
the  vocabulary  can  be  "understood"  by  a computer,  and  there  is  no  doubt  that 
it  can,  then  an  important  commun icition  link  can  be  established  between  the 
operator  and  his  equipment.  At  least  three  advantages  are  apparent  in  such 
a communication  channel. 

1.  As  the  ERPs  seem  to , at  least  in  part,  manifest  events  in  the  non- 
verbalized  substrate  of  cognition  they  may  provide  information  which 
cannot  be  otherwise  communicated  by  the  operator. 

2.  The  immediate  access  ERP  components  provide  into  the  subject's  cogni- 
tive world  suggest  that  they  might  enhance  communication  speed. 

3.  The  strong  presumptive  evidence  that  an  individual's  ERP  is  unique 
suggests  that  these  potentials  might  constitute  an  ongoing  self 
authenticating  component  of  the  communication  prodess. 

The  project  plan  proposed  here  capitalizes  on  the  success  of  the  first 
two  years  of  our  ARPA  support  and  is  designed  to  develop  realistic  implemen- 
tations of  biocybernetic  comm  mication  systems  in  simulated  operational  en- 
vironments. 

We  see  the  following  specific  tasks. 

a.  Refine  our  present  knowledge  of  the  ERP  vocabulary.  Particular 


emphasis  will  be  placed  on  the  newly  discovered  N190  and  "slow  wave" 


components , as  well  as  on  consolidating  our  understanding  of  P300 
as  a measure  of  the  feedback  subjects  derive  from  specific  stimuli. 

b.  lest  the  utility  of  the  well  established  vocabulary  in  enhancing 
performance  of  operators  controlling  complex  devices  (with  ovr 
major  model  being  the  aircraft  cockpit) . 

c.  Test  the  utility  of  the  vocabulary  in  the  Computer  Assisted  Instruc- 
tion  environment.  Here  the  model  will  be  the  PLATO  lesson  and  our 
goal  the  facilitation  of  interaction  between  the  student  and  the 
teaching  console. 

The  major  difficulty  in  developing  the  ERP  as  a communication  channel 
is  the  very  low  signal  to  noise  ratio  in  the  channel.  ^ The  ERPs  are  minute 
m amplitude  relative  to  the  "polyneural  -oar"  of  the  ongoing  EEG  activity. 

In  determining  the  vocabulary , signal  to  noise  ratios  can  be  enhanced  through 
the  use  of  signal-averaging.  This  technique  relies,  of  course,  on  the  re- 
peated occurrence  of  the  cognitive  events  as  well  as  on  the  computer ' s pre- 
cise knowledge  of  their  time  of  occurrence.  While  this  procedure  is  adequate 
for  vocabula.-y  identification  it  will  not  do  for  practical  communication  in 
operational  environments . In  these  we  must  be  able  to  identify  the  occurrence 
of  ERP  components  in  the  ongoing  EEG  activitu  immediately  following  sinc/le 
occurrences  of__the  specific  events.  The  problem  then  becomes  one  of  developing 
single-trial  signal  extraction  techniques  (STSE) . 

We  have  bean  developing  STSE  procedures  for  some  time . ^ Work  conducted 
with  our  present  ARPA  support  indicates  that  Stepwise  Discriminant  Analysis 
(SWDA)  procedures  allow  correct  classification  of  trials  at  a hit  rate  of 
80  percent.  This  hit  rate  is  based  on  a crude  application  of  SWDA.  It  is 
likely  that  STSE  could  be  greatly  improved  by  combining  data  from  multiple 
channels  and  by  prefiltering  the  ongoing  EEG  activity  to  eliminate  energy  in 
the  "noise"  bandwidth  before  discrimination  begins.  Development  work  on  STSE 
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could  not  begin  in  earnest  prior  to  the  identification  of  the  vocabulary  and 
it  is  contingent  on  computational  power  which  will  be  at  our  disposal  accord 
ing  to  present  project  plans.  We  intend  to  devote  much  effort  to  the  single 
trial  problem.  The  development  of  a STSE  is  contingent  also  on  the  develop- 
ment of  refined  techniques  to  increase  signal  to  noise  ratio  in  the  real- 
time evaluation  of  performance.  In  fact,  failures  to  "correctly"  detect 
ERPs  on  single  triads  may  often  be  due  to  variability  in  the  concomitant 
performance  variables  as  much  as  it  is  due  to  inherent  "noise"  in  the  EEG. 
Thus  the  development  of  STSE  techniques  requires  a conjoint  measurement  of 
performance  and  EEG  variables.  We  need  techniques  which  lead  towards 
sufficiently  accurate  definitions  and  measures  of  single  "performance" 
events,  which  can  then  be  correlated  with  their  ERP  counterparts.  The 
investigation  of  this  problem  and  the  development  of  such  techniques  will 
be  directed  by  the  Aviation  Research  Laboratory.  Both  research  programs 
will  be  joined  in  an  effort  to  improve  the  real-time  evaluation  of  cogni- 
tive and  behavioral  events.  All  data  collected  in  all  experiments  to  be 
conducted  in  the  laboratory  would  provide  a data  base  for  the  development 
of  STSE  procedures . The  operational  environments  in  turn  will  be  used  for 
specific  demonstrations  of  these  procedures  as  they  emerge. 

2.  Research  Plan 

Three  concurrent  activity  streams  will  be  maintained  during  the  next 
year. 

a.  The  procurement  and  installation  of  hardware  and  software  for  re- 
cording ERPs  in  simulated  cockpits  and  in  a CAI  installation. 

b.  Intensive  development  of  STSE  techniques  and  the  implementation  of 
at  least  one  demonstration  in  which  a STSE  is  used  in  biocybernetic 
communi cation. 

c.  Continuation  of  work  on  vocabulary  identification,  using  our 
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existing  facility . 

It  should  be  emphasized  that  even  though  a major  development  effort  will 
be  required  by  activity  stream  a ve  plan  to  maintain  a steady  level  of  effort 
in  the  other  two  categories.  If  the  proposed  personnel  contingent  is  approved 
the  installation  of  the  new  laboratory  can  be  accomplished  with  no  reduction 
in  our  present  rate  or  progress . Moreover,  we  intend  to  support  the  use  of 
our  facilities  by  remote  investigators  as  the  need  arises. 

We  shall  now  describe  briefly  specific  aims  and  goals  of  the  three 
activity  streams. 

2 . 1 The  development  of  a Performance  lab. 

The  present  project  will  be  conducted  in  close  collaboration  with  the 
Aviation  Research  Laboratory  at  the  University  of  Illinois  (ARL) . The  rapid 
development  in  aircraft  technology  during  recent  years  has  increased  the 
emphasis  on  the  assessment  of  human  capacity  with  regard  to  attention  capabili- 
ties and  information  processing.  The  emerging  cockpit  typically  requires 
man  to  be  an  information  manager  or  fast  decision  maker  rather  than  a direct 
controller  of  flight  variables . In  the  past  several  years  the  ARL  has  con- 
ducted a comprehensive  program  directed  among  other  things  at  the  development 
of  general  rules  and  prediction  equations  for  the  evaluation  of  task  load  and 
operator  efficiency . In  particular  this  research  has  developed  a new  metho- 
dological approach  for  measuring  operator  1 >ad , as  well  as  individual  differ- 
ences in  attention  capabilities.  The  method  incorporates  adaptive  techniques, 
sensitive  feedback  devices  and  on  line  evaluation  of  operator  performance . 

We  intend  to  construct  in  collaboration  with  ARL  a laboratory  which  would 
allow  recording  of  ERR  in  an  environment  in  which  the  performance  evaluation 
methodology  is  implemented.  This  will  require,  in  addition  to  our  standard 
computer-based  electrophysiologicc  1 data  acquisition  system,  arrangements 
for  presenting  audio-visual  displays  of  considerable  complexity  while  subjects 
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manipulate  control  gear.  The  computer  supporting  this  facility  should  be 
capable  of  generating  on-line , and  in  roai  time,  tasks  of  realistic  complexity. 

By  the  end  the  first  year  of  the  project  we  will  accomplish  the  fol- 
lowing: 

a.  Design,  procure  and  install  the  laboratory  facility  described  above. 

b.  Develop  procedures  for  routine  recording  of  BEG,  and  ERPs , while 
subjects  perform  complex  control  tasks. 

c.  Begin  an  experimental  program  designed  to  determine  if  ERP  components 
such  as  N190,  P300  or  the  CNV  behave  in  the  operational  environment 
m tne  same  manner  they  behave  in  the  simpler  lab  situations.  This, 
for  example,  will  include  the  following  experiments . 

c.l  Determine  the  utility  of  N190  ERP  component  as  a measure  of 
reserve  capacity,  by  recording  N190  to  irrelevant  but  rare 
stimuli  at  carefully  controlled  levels  of  task  difficulty. 
c.2  Determine  the  utility  of  P300  as  a tool  in  evaluating  the 
scope  and  content  of  the  focus  of  attention,  by  presenting 
subjects  with  a very  complex  display,  different  elements  of 
which  may  be  relevant  to  the  subject's  task  at  any  time. 
c.3  Determine  if  any  ERP  component  can  be  used  to  communicate  to 
complex  display  system  which  of  many  possible  displays  a sub- 
ject requires  to  optimize  task  performance . 

d.  Develop  in  the  Aviation  Research  Laboratory  a capability  for  recording 
ERPs  from  subjects  operating  the  GAT  cockpit  simulator.  During  the 
first  year  this  will  require  the  installat  on  of  physiological  re- 
cording equipment.  By  the  third  year,  however,  a full  scale  computer- 
ized data  acquisition  system  will  be  developed  at  ARL. 
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2 ^ Z2Z£  development  of  a CAI  environment  for  ERP  testing. 

The  use  of  ERP  components  in  man  machine  communication  is  feasible 
only  when  computer  power  is  available  in  the  system.  The  student-console 
interaction  which  characterizes  computer  assisted  instruction  is  therefore 
a natural  candidate  environment  for  our  tests,  as  in  the  performance-enhance- 
ment. task  individuals  must  exchange  information  with  a computing  system.  The 
goal  here,  of  course,  is  to  modify  the  information  store  of  the  operator 
rather  than  to  provide  ''efficient"  control  of  the  console.  Yet,  the  trans- 
actions between  the  student  and  his  "teacher"  need  to  be  so  designed  that 
the  computer’s  resources  are  utilized  optimally  to  enhance  the  student's 
understanding  and  retention  of  the  material . In  this  context  it  is  con- 
ceivable that  valuable  information  about  the  student's  interaction  with 
the  computer  can  be  obtained  through  the  use  of  ERP  components.  With  ERPs 
the  Instructor"  might  seek  to  find  out  how  surprised  was  the  student  when 
an  answer  he  gave  turned  out  to  be  correct  (or  incorrect) . How  helpful 
was  the  additional  information  just  supplied  in  reducing  the  student's 
uncertainty  about  his  choices?  How  ready  is  the  student  to  respond? 

The  use  of  ERPs  in  the  CAI  environment  is  somewhat  less  straightforward 
than  their  use  in  the  performance  enhancement  area,  where  perfoimance  en- 
hancement can  be  rigorously  defined  and  measured.  The  tangible  effects  of 
instructional  improvements  are  always  difficult  to  describe  and  to  measure. 

In  fact,  it  might  very  well  be  the  case  that  ERPs  will  contribute  to  this 
field  through  the  provision  of  such  criteria. 

To  fully  implement  a research  program  into  use  of  ERPs  in  the  CAI  en- 
vironment it  will  be  necessary  to  develop  a laboratory  in  which  the  PLATO 
terminal  na capabilities  not  normally  available  in  the  standard  PLATO  ter- 
minal. The  "intelligent"  terminal  should  allow  for  large  scale  exchange  of 
data,  and  synchronizing  signals,  between  our  laboratory  computer  and  the 
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PLATO  system.  Moreover,  we  should  have  faster  and  more  sophisticated  graphic 
capabilities  within  the  PLATO  lessons  to  allow  fine  control  of  the  stimulus 
parameters  used  in  invoking  the  ERPs.  We  also  anticipate  a need  for  inter- 
change of  data  between  PLATO,  our  lab  system  and  the  ARPANET.  All  of  the 
above  capabilities  are  provided  by  the  Intelligent  PLATO  Terminal  developed 
under  the  direction  of  Professor  Roger  Johnson  of  Computer  Assisted  Educa- 
tion Research  Laboratory  at  the  University  of  Illinois . One  of  the  prime 
tasks  therefore , during  the  first  project  period , would  be  the  installation 
of  the  PDP/11  based  PLATO  terminals  and  its  interfacing  to  a physiological 
recording  system.  As  soon  as  possible  we  will  begin  to  develop  techniques 
for  measuring  ERPs  associated  with  specific  events  .in  the  PLATO  lesson. 

Thus,  for  example,  we  shall  try  to  record  ERPs  elicited  by  the  "arrow" 
(response  solicitor)  in  the  PLATO  frame,  or  by  the  "OK"  vs  "error"  indica- 
tors following  a student's  response.  Cortical  activity  preceding  the  "NEXT" 
button  press.  We  are  currently  arranging  to  install  a standard  PLATO  ter- 
minal in  our  laboratory . Immediately  upon  its  installation  we  will  begin 
developing  a repertoire  of  CAI-ERP  procedures  on  which  basis  a detailed 
experimental  program  could  be  developed . 

During  the  first  year  we  shall  identify  classes  of  events  in  a CAI  les- 
son stream  which  are  reliably  associated  with  ERPs.  Several  specific  experi- 
ments will  be  carried  out  to  assess  the  meaning  of  the  ERP  vocabulary  in 
a CAI  context.  In  parallel,  we  will  install  the  advanced  PDP/11  based 
PLATO  termina’.  By  the  time  it  is  operational  the  preliminary  testing 
with  the  standard  terminal  will  be  completed  and  a full  scale  program  would 
be  underway . 

2 . 3 Methodology : Sinqi e trial  extraction  techniques. 

Using  data  currently  on  hand,  as  well  as  data  collected  during  the  pro- 
iec t period,  we  shall  continue  our  development  of  STSE  procedures . The 


following  illustrates  the  projects  to  be  undertaken. 

a.  Evaluate  discriminant  functions  developed  on  the  basis  of  a single 
recording  site.  This  work  is  on-going.  The  functions  we  have  are 
quite  powerful.  We  will  continue  to  validate  the  available  func- 
tions against  behavioral  data  and  we  will  develop  at  the  same  time 

software  required  to  move  as  much  of  the  required  computation  on- 
line as  possible . 

b.  The  single  electrode  techniques  will  be  extended  to  incorporate 
data  from  several  channels  (electrodes) . This  will  clearly  im- 
prove the  power  of  the  discriminant  function. 

c.  Use  the  autoregressive  procedure  developed  in  our  laboratory  by 
Ron  Herning  to  eliminate  the  "noise"  from  the  post-event  EEG , in. 
an  attempt  to  enhance  the  power  of  the  discriminant  functions . 

d.  Continue  to  test  various  non-linear  and  clustering  algorithms  as 
alternates  to  linear  discriminant  analysis. 

e.  Continue  the  development  of  factor-analytic  techniques  in  the  dis- 
sociation of  ERP  components.  In  particular,  focus  on  procedures 
that  would  allow  the  use  of  factor-scores  of  single  trials  in  on- 
line discrimination. 

f.  Considerable  emphasis  will  be  placed  on  the  development  of  cross- 
subject  discriminant  functions.  Alternately,  we  will  search  for 
techniques  which  will  allow  the  adjustment  of  general  functi „ ’s 
to  data  from  individual  subjects . The  scenario  we  have  in  mind 
is  one  in  which  a universal  function  is  developed  off-line , yet 
for  each  subject  a brief  "calibrating"  on-line  sequence  will  allow 
a full  adjustment  of  the  universal  function  to  his  idiosyncratic 
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Successful , and  cost  effective , implementation  of  this  plan  is  con- 
tingent on  the  availability  of  a system  tnat  would  allow  the  management , 
and  ready  processing,  of  rather  large  data  bases.  We  need  a combination 
of  local  computing  power,  a sophisticated  data  management  facility  and 
eFficient,  reliable,  access  to  remote  number-crunching  machines.  The 
PDP11/70  will  provide  all  of  this  in  addition  to  its  role  as  the.  central 
node  in  the  network  of  laboratory  computers  that  will  control  the  experi- 
ments and  manage  the  data  acquisition. 

2 . 4 ERP  vocabulary  identification. 

This  activity  stream  is  a direct  continuation  of  work  currently  con- 
ducted in  our  lab  with  ARPA  support.  Many  experiments  are  planned.  The 
following  can  serve  as  illustration. 

a.  Does  P300  depend  on  the  feedback  qualities  of  the  eliciting 
stimuli?  Subjects  perform  a "motor"  task  and  are  informed  of 
their  success  or  failure.  It  turns  out  the  P300  amplitude  for 
the  ERP  elicited  by  the  correct  indicator  depends  on  the  differ- 
ence in  intensity  between  the  "correct"  and  the  "error"  indicator. 
The  precise  conditions  under  which  this  interaction  occurs  must 
be  determined. 

b ' Data  we  are  collecting  now  suggests  that  the  amplitude  of  P300 
depends  on  the  rate  at  which  the  stimuli  occur.  A parametric 
study  of  this  effect  is  clearly  called  for. 

c.  The  "novelty"  component,  (N190) , must  be  studied  in  great  detail. 
While  the  present  data  suggest  that  it  is  enhanced  in  EPs  elicited 
by  rare,  irrelevant  stimuli,  tiese  conclusions  must  be  tested  in 
other  circumstances . 

d.  The  relation  between  the  Readiness  Potential  and  the  CNV  must  be 
cleared  up  if  we  are  to  be  able  to  infer  with  precision  the  subjects' 
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intentions"  from  anticipatory  waves.  We  believe  that  the  CNV 
represents  a general  preparatory  process  which  is  maximal  over 
the  area  uf  the  brain  which  needs  to  be  prepared  for  a given 
task.  Tests  of  the  laterality  of  the  CNV  under  conditions  of 
motor  and  cognitive  preparation  will  be  conducted, 
c.  If  the  ^ NV  represents , as  many  believe,  a preparatory  process 
generated  m the  brain's  attempt  to  optimize  performance,  then 
there  ought  to  be  a high  (negative)  correlation  between  CNV  ampli- 
tude and  reaction  time.  The  .literature  suggests  that  this  is  not 
the  case.  We  feel  that  the  present  literature  did  not  do  justice 
to  the  problem  because  it  treated  the  Reaction  Time  as  a uniform 
variable.  Reaction  time  in  fact  is  determined  by  a variety  of 
stimulus  intake,  information  processing , and  response  selection 
processes.  We  plan  a series  of  studies  that  will  dissociate 
Reaction  time  into  its  constituents  and  determine  to  which  of 
these  constituents  L!RP  components  relate.  This  will  present  a 
major,  and  novel,  deepening  of  our  understanding  of  the  ERP 
vocabulary . 

f.  we  are  now  studying  the  degree  to  which  P300  amplitude  reflects 
the  salience  of  elements  of  a compound  stimulus.  We  discovered 
that  visual  and  auditory  P300  can  be  clearly  discriminated  and 
that  their  relative  amplitude  seems  to  reflect  the  salience  of 

the  modality.  We  p i to  xt end  our  investigations  of  this  rather 
powerful  procedure. 

g.  The  recent  development  of  a relatively  inexpensive  speech  synthe- 
sizer - the  Votrax  - which  can  drive  our  PDP/11  +o  generate 
precisely  defined  verbal  stimuli  makes  it  feasible  to  broaden 

the  dimensions  of  stimulus  materials  we  use  in  P300  studies.  We 
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plan  to  acquire  a Votrax  and  use  it  to  determine  if  P300  can  be 
elicited  by  "unexpected"  stimuli,-  when  the  oddity  of  the  stimuli 
is  controlled  through  varying  semantic  and  linguistic  dimensions . 

This  is  but  a small  sample  of  the  experiments  which  we  would  like  to 
run  as  part  of  activity  stream  C.  Needless  to  say  these  studies  will  pro- 
vide data  for  testing  and  developing  STSE  techniques  as  well  as  provide 
models  for  the  experiments  conducted  in  the  performance  and  CAT  setting. 

3 . Facilities  Available 

This  project  will  be  conducted  in  the  Evoked  Potential  Laboratory  (EPL) 

in  the  Department  of  Psychology  as  well  as  in  the  Aviation  Research  Labora- 

2 

tor y.  The  EPL  currently  occupies  1182  ft  of  laboratory  space,  (exclusive 
of  office  space  allocated  to  our  use) . The  Department  of  Psychology  is 
ready  to  commit  this  space  for  our  use  j.or  the  duration  of  the  project.  The 
Department  has  also  recognized  thee  the  full  complement  of  laboratories  and 
facilities  called  for  in  this  proposal  will  require  the  commitment  of  sub- 
stantial additional  space.  It  is  a measure  of  the  support  we  have  in  the 
Department  that  additional  space  ranging  up  to  700  ft'  of  laboratory  space 
will  be  allocated  to  this  project.  Office  space  for  project  personnel  will 
also  be  provided  in  the  Psychology  Building. 

While  the  Psychology  Building  is  a splendid  structure,  built  in  1968, 
and  is  exceptionally  well  designed  to  support  a large,  research  oriented , 
department , the  specific  needs  of  this  project  will  call  for  several,  sub- 
stantial, renovations , All  will  be  in  support  of  the  multiple  computer 
environment  we  are  planning . Specifically  the  need  will  arise  for  (a)  in- 
stalling sufficient  electrical  power  to  support  the  computers , ib)  installing 
a raised  floor  in  the  room  that  will  house  the  PDP11/70  and  the  PDPll/40 
and  (c)  providing  additional  airccnditioning  and  humidity  control  for  the 
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computcr  room.  Estimating  the  precise  cost  o£  these  renovations  requires 
engineering  ct'frrt.  Gross  estimates  made  by  our  Physical  Plant  engineers 
are  that  ihe  total  cost  will  probably  be  in  the  neighborhood  of  $50,000. 

The  University  has  undertaken  to  introduce  the  required  changes,  and  to 
charge  no  more  than  $50,000.  The  University  will  cover  all  cost  over~ runs 
which  exceed  this  estimate.  Tf  the  renovations  arc  less  costly , the  allo- 
cated funds  will  remain  in  the  contract  for  use  as  directed  by  ARPA,  or 
applied  against  other  expenses. 

The  Aviation  Research  Laboratory , Institute  of  Aviation,  University 
of  Illinois  offer  unique  facilities  to  support  aviation  research  in  the 
areas  of  systems  design,  personnel  selection  and  training.  The  Institute 
has  a fleet  of  approximately  40  training  aircraft  and  6 training  simulators . 
The  Laboratory  facilities  include  a Specialized  Flight  Simulation  Facility 
consisting  of  £:  Singer-Link  GAT-2  simulator  and  a Raytheon  model  704  digital 
computer , a Digital  Equipment  Corporation  model  PDP11/40  digital  computer 
and  a terminal  on  PLATO  IV,  the  University's  CAI  system. 

The  GAT-2  simulator  consists  of  a self  contained  computer,  two  place 
cockp-t  and  a two  axis  motion  system.  Its  motion  system,  flight  controls 
and  instrument  panel  have  been  extensively  modified  to  satisfy  requirements 
of  the  research  programs . An  external  visual  display  for  presentation  of 
computer  generated  graphics  by  means  of  a television  data  link  is  currently 
being  installed.  In  addition,  it  has  been  interfaced  to  the  Raytl^eon  704 
digital  computer  for  auxiliary  computations  and  automated  data  collection. 

ihe  two  computers  are  connected  by  means  of  a high  speed  interface . 

Each  computer  includes  a compliment  of  peripheral  devices  as  well  as  signal 
conversion  equipment  and  a DIGIVUE  plasma  panel  with  a touch  input  feature. 
Specialized  hardware  for  alpha-numeric  symbol  venerations , random  number 
generation,  matrix  multiplication  and  computtr  generated  speech  is  also 
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included . The  PDPll/40  is  connected  by  means  of  a telephone  line  to  the 
DEC  10  located  in  the  Coordinated  Sciences  Laboratory . 

During  the  first  two  years  of  the  project  the  bulk  of  the  activity 
will  be  at  the  EPL.  However,  we  are  planning  to  begin  recording  ERPs 
within  the  simulated  cockpit' s of  ARL  within  the  third  year  of  the  project . 
The  budgetary  implications  of  this  plan  will  be  described  below. 
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ANNOTATED  LIST  OF  PUBLICATIONS  OF  RESEARCH  SUPPORTED  BY  ARPA 


A.  General  Reviews 


A.l  Donchin,  E.  Brain  electrical  correlates  of  pattern  recognition. 

In  Signal  Analysis  and  Pattern  Recognition  in  Biomedical 
Engineering , (Inbar,  G.,  Ed.).  John  Wiley,  1975,  199-218. 

This  overview  of  our  approach  to  the  study  of  FRPs  as  an  indicant 
of  cognitive  activity  was  presented,  by  invitation , before  an  Interna- 
tional Symposium  on  Signal  Analysis  and  Pattern  Recognition  held  in 
Haifa,  Isiael  in  1973.  It  was  intended  for  an  audience  of  non-specialists 
and  it  therefore  prrsents  a general  introduction  to  evoked  potentials 
in  general  as  well  as  to  the  work  of  this  laboratory . 


A. 2 Donchin,  E.  (Ed.)  The  relationship  between  P300  and  the  CNV.  A 
correspondence  and  an  experi mental  report.  In  Event  Related  Slow 
Potentials  of  the  Brain  (NcCallum,  C.  and  Knott,  J.,  Eds.) 

roceedings  of  the  3rd  International  Congress  on  CNV  and  Slow 
Potentials,  Bristu. , England,  August  1973  (in  press). 

A detailed  discussion  of  the  relai'on  between  the  CNV  and  P300. 

This  report  bears  directly  on  the  elucidation  of  the  ERP  vocabulary . 

It  is  in  fact  the  edited  transcript  of  a mail  correspondence  workshop 
held  during  1973.  It  illustrates  the  conceptual  and  methodological  pro- 
blems that  are  encountered  in  an  attempt  to  deal  with  vocabulary  problems . 


A. 3 Donchin,  E.  (Ed.)  Publication  r-iteria  for  studies  of  evoked  poten- 
tials (EP) . Report  of  the  Methodology  Committee.  In  Proceedings 
of  the  International  Symposium  on  Cerebri  1 Evoked  Potcn tials  in 
Nan  (Desmedt,  J.,  Ed.)  Brussels,  April  1974  (in  press). 

The  report  on  an  inte' national  committee  organized  and  chaired  by 

E.  Donchin  on  the  proper  procedures  to  be  used  in  ERP  research . 
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A.4  Donchin,  E.  and  Hefflcy,  E.  Using  minicomputers  in  the  signal 

averaging  laboratory . American  Psychologist , 1975,  30:  299- 
312. 

A description  of  the  laboratory  developed  with  ARPA's  support.  A 
detailed  discussion  of  the  hardware  and  software  required  for  the  suc- 
cessful implementation  of  ERP  research  is  included. 


B.  Measurement  Problems  - The  STSE  Technique 

B.l  Donchin,  E.  Measurement  problems  in  slow  potential  research  ( Open 
ing  remarks).  In  Event  Related  Slow  Potentials  of  the  Brain 
(McCallum,  C and  Knott,  J.,  Eds.)  Proceedings  of  the  3rd 
International  Congress  on  CNV  and  Slow  Potentials,  Bristol , 
England,  August  1973,  in  press. 

A brief  review  of  the  application  of  discriminant  analysis  to  ERP 


data . 


B.2  Donchin,  E.  and  Earning,  R.  I.  A simulation  study  of  the  efficacy 

of  stepwise  discriminant  analysis  in  the  detection  and  compari- 
son of  event  related  potentials.  Electroencephalography  and 
Clinical  Neurophysiology , 1974,  38:  51-68. 

A detailed  analysis  of  the  efficacy  of  stepwise  discriminant  analysis 
when  applied  to  ERP  data.  The  final  section  of  this  paper  presents  a 
demonstration  of  the  approach  we  plan  to  use  in  the  on-line  detection  of 
"single-trial"  cognitive  events. 

B.3  Herning,  R.  I.  and  Donchin,  E.  The  use  of  ERPs  in  the  assessment  of 
sensory  sensitivity . (in  preparation) 

This  report,  now  in  the  final  writing  stages,  describes  an  extensive 
test  of  the  procedures  outlined  in  the  final  section  of  Donchin  and 
Herning  (B.2)  using  auditory  evoked  responses . The  same  data  base  has 
been  used  for  Herning' s dissertation  study  on  the  application  of  auto- 
regressive techniques  to  single  trial  detection. 
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3.4  Squires,  K.  and  Donchin,  E.  Beyond  averaging : The  use  of 

discriminant  functions  to  recognize  event  related  potentials 
elicited  by  single  auditory  stimuli.  Electroencephalography 
and  Clinical  Neurophysiology , 1976  (in  press) . 

A demonstration  of  the  efficacy  of  stepwise  discriminant  analysis 
in  the  detection  of  the  P300  component  of  the  ERP . 

C . Studies  Related  to  the  Readiness  Potential 

C.l  Kutas , M.  and  Donchii , E.  Studies  of  squeezing : The  effects  of 

handedness , the  responding  hand,  and  response  force  on  the 
contralateral  dominance  of  the  readiness  potential . Science, 
1974,  186(4163):  545-548. 

A preliminary  report  on  the  relationship  between  the  responding 
hand,  handedness  and  premotor  negative  waves.  Study  was  important  in 
demonstrating  the  need  for  analyzing  the  multiple  behavioral  deter- 
minants of  ERP  components. 


C.2  Kutas,  M.  and  Donchin,  E.  The  effect  of  handedness , the  responding 
hand,  ind  response  force  on  the  contralateral  dominance  of  the 
"readiness"  potential.  In  Proceedings  of  the  International 
Symposium  on  Cerebral  Evoked  Potentials  in  Nan  (Desmedt , J., 

Ed.)  Brussels,  April,  1974  (in  press). 

A very  detailed  report  of  the  study  reported  in  Kutas  and  Donchin  (C.l). 


C.3  Kutas,  M.,  McCarthy , G.,  and  Donchin,  E.  Differences  between 

sinistrals'  and  dextrals'  ability  to  infer  a whole  from  its  parts: 
A failure  to  replicate.  Neuropsychologia , 1975,  13:  455-464. 

An  analysis  of  various  handedness  tests  undertaken  as  a preliminary 

to  further  elucidation  of  the  relation  between  the  hand  in  use  and 


handednt.  ss . 


C.4  Donchin,  E.,  Kutas,  M. , and  McCarthy,  G.  Comparison  of  the  hemis- 
pheric asymmetries  of  the  "Readiness  Potential"  and  the  "CNV." 
Paper  road  before  the  Psychonomic  Society  15th  Annual  Meeting , 
Boston,  Massachusetts , Noveml)er,  1974. 
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A detailed  publication  bused  on  the  study  reported  in  this  brief 
meeting  presentation  is  now  in  writing.  The  study  suggests  that  the 
scalp  distribution  of  negative  "expectancy"  waves  varies  with  the 
specific  nature  of  the  expectancy . 


C.5  Donchin,  E. , Kutas,  M. , and  McCarthy,  G.  Electrocortical  indices 
of  hemispheric  utilization.  In  Lateralization  in  the  Nervous 
System  (Harnad , S. , Ed.)  New  York,  Academic  Press,  in  press. 

A review  of  the  literature , a methodological  critique  of  the 

literature  and  a report  of  several  of  our  own  studies. 


D . Studies  in  the  ERP  Vocabulary 

D.l  Donchin,  E. , Johnson,  R.,  Herning , R.  I.,  and  Kutas,  M.  Covariation 
of  the  magnitude  of  the  CNV  and  P300  as  a function  of  the  sub- 
ject’s task.  In  Event  Related  Slow  Potentials  of  the  Brain 
(McCallum,  C.  and  Knott,  J. , Eds.)  Proceedings  of  the  3rd 
International  Congress  on  CNV  and  Slow  Potentials,  Bristol, 
England,  August  1973,  in  press. 

A demonstration  that  P300  is  not  a dichotomous  phenomenon,  but 
rather  varies  as  a function  of  subject  task.  Preliminary  data  presented 
on  the  differential  scalp  distributions  of  the  CNV  and  P300 . 


D.2  Donchin,  E. , Tueting , P. , Ritter,  W. , Kutas,  M.,  and  Neff ley,  E. 

On  the  independence  of  the  CNV  and  the  P300  components  of  the 
human  averaged  evoked  potential.  Electroencephalography  and 
Clinical  Neurophysiology , 1975,  38:  449-461. 

This  study,  which  follows  up  on  Donchin  et  al.'s  discussion  (A. 2) 
has  definitively  demonstrated  that  the  CNV  and  P300  are  independently 
controlled  ERP  components , and  that  P300  is  independent  of  prior  expec- 
tancies. This  paper  also  describes  in  some  detail  the  use  we  make  of 
a combination  of  Principal  Component  Analysis  and  Analysis  of  Variance 
in  measuring  ERP  components . 
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D.3  Squires,  K.  C .,  Donchin,  E.,  Herning , R.  I.  , and  McCarthy,  G. 

On  the  influence  of  task  relevance  and  stimulus  probability 
on  ERP  components.  Electroencephalography  and  Clinical 
Neurophysiology , in  press.  ~ ‘ 

This  study  demonstrates  the  existence  of  multiple  endogenous  com- 
ponents in  the  190  - 500  ms  latency  range.  The  paper  is  important  in 
that  it  delineates  the  variables  affecting  P300  as  well  as  reporting 
the  discovery  of  N190,  a component  apparently  related  to  "novelty"  (?) 
independently  of  the  suhlcrt' s task 

! 

i 

Copies  of  the  above  manuscripts  can  be  obtained  by  writing  E.  Donchin, 

Department  of  Psychology,  University  of  Illinois,  Champaign , Illinois  61820. 
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EX  PER  I MEN TAL  PLANS  DEVELOPED  AS  A RESULT  OF  THIS  PROJECT 
1 . Human  Performance  Studies 

The  attempt  to  develop  a system  for  the  enhancement  of  human  performance 
through  man-machine  interaction  via  an  ERP  link  will  proceed  along  several 
separate  but  not  independent  lines.  Because  of  the  potential  richness  of 
the  information  available  in  the  form  of  separable  components  within  the 
ERP,  more  than  one  aspect  of  the  subject's  state  may  be  assessed  from  each 
stimulus  presentation  or  from  a comparison  of  the  ERPs  presented  in  several 
modalities  within  the  same  t>eriod  of  time.  Thus  it  may  be  possible,  within 
a task,  to  monitor  '.verall  level  of  performance , the  relative  division  of 
attention  among  sets  of  stimuli , and  the  amount  of  information  or  utiliza- 
tion of  information  within  an  attended  channel. 

The  end  product  of  the  research  will  be  a system  which  utilizes  all  of 
the  information  available  from  the  ERP.  .Initial  experiments  will,  of  neces- 
sity, be  directed  at  specific  aspects  of  the  ERP  and  what  they  reflect. 

The  major  emphasis  in  ERP  research  has  been  on  components  which  are 
associated  with  the  processing  of  signal  information  by  a subject,  thus  re- 
quiring that  signals  be  relevant  to  a required  task.  In  such  a situation 
a P300  component  is  commonly  reported.  When  signals  are  irrelevant  to  the 
subject  (for  example,  when  repetitive  identical  stimuli  are  presented  with 
no  task  requirement) , the  P300  is  absent  and  a simple  negative-positive 
( N100-P200 ) wave  is  seen.  We  have  found  that  under  some  circumstances  an 
additional  component  (N190)  is  also  present  in  the  waveforms  of  ERPs  eli- 
cited by  unattended  stimuli  (Squires,  Donch.in , Homing , and  McCarthy /sub- 
mitted for  publication) . This  occurs  when  a "rare"  stimulus  is  inserted  in 
a regular  train  of  stimuli,  even  when  those  rare  stimuli  have  no  apparent 
relevance.  Certainly , no  overt  notice  is  taken  by  the  subjects  of  the 
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ctimuli.  Thus  we  infer  that  at  some  level  of  the  processing  of  the  signal 
input  the  stimulus  change  ha:  been  noted,  and  reflected  in  t ERP,  but 
the  processing  stops  at  a level  prior  to  that  associated  with  P300  since  a 
P300  component  is  not  elicited . 

As  N190  has  just  been  discovered  a systematic  study  of  its  associations 
is  get  to  be  performed.  One  plausible  hypothesis  regarding  11190  is  that 
it  may  index  the  capability  of  a person  co  react  to  task-irrelevant  stimuli 
while  performing  in  a required  primary  task . If  so,  since  it  requires  no 
activity  on  the  subject's  part,  the  N190  may  be  a potentially  useful  tool 
for  assessing  the  subject's  state  without  interfering  with  other  activities 
that  might  be  required  of  the  subject. 

Investigation  of  these  hypotheses  requires  the  comparison  of  ERP  com- 
ponents while  manipulating  the  amount  of  processing  and  response  load  im- 
posed on  the  subject.  The  power  of  such  a comparison  is  therefore  depen- 
dent not  only  on  a refined  ability  to  detect  ERP  events,  but  also  on  the 
existence  of  appropriate  teclviiques  to  evaluate  performance  capacity  and 
task  load. 

The  Aviation  Research  Laboratory  has  devoted  intensive  efforts  to 
develop  procedures  that  will  enable  the  scaling  of  task  levels  in  terms 
of  operator  capacity  and  task  load.  These  procedures  were  developed  both 
in  focused  attention  - single  task  conditions  and  divided-attention  time- 
sharing situations.  Adaptive  techniques  and  elaborated  feedback  devices 
are  employed  to  adjust  task  difficulty  and  manipulate  demands  rescaled  to 
correspond  with  the  attention  capacity  of  each  subject. 

In  the  first  series  of  experiments  ERP  will  be  measured  while  subjects 
perform  a dual-axis , pursuit,  tracking  task.  The  subject  will  attempt  to 
track  a random  noise  forcing  function  through  a spring-centered  dual-axis 
hand  ountroller . Task  difficulty  will  be  manipulated  in  one  of  four  ways: 
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(1)  Adaptive  change  of  forcing  function  frecjuency  (by  changing  the 
upper  limit  of  the  low  pass  filter).  This  represents  a stimulus 
variable. 

(2)  Adaptive  change  of  control  dynamic  (by  changing  the  relative 
portions  of  first  and  second  order  determinants) . This  repre- 
sents a response  variable . 

(3)  Changing  the  tolerance  limits  of  tracking  errors.  In  this  condi- 
tion all  other  variables  are  fixed  and  performance  bar  graphs 
are  employed  to  indicate  the  instantaneous  differences  between 
actual  and  desired  performance . 

(4)  In  addition  to  the  above  provisions , the  program  also  allows 
selection  of  single  or  dual  axis  manipulation. 

Subject's  performance  will  be  recc.ded  and  analyzed  on  line. 

ERPs  will  be  recorded  in  response  to  unobtrusive  . epetitive  tone  bursts 
which  will  be  continually  present  and  of  no  information  value  within  the 
task.  Occasionally  one  of  the  tone  bursts  will  be  of  a different  frequen- 
cy or  intensity  from  the  others  and  will  elicit  a waveforr i characterized  by 
an  N190  component.  Several  levels  of  task  involvement  will  be  tested 
ranging  from  performing  the  task  at  a very  easy  level  up  i.o  the  maximum 
level  the  subject  is  capable  of,  along  with  the  appropriate  control  condi- 
tions . 

Data  analysis  will  be  based  on  our  standard  factor  analytic  procedures 
which  allow  an  examination  of  ERP  components  as  a function  of  the  performance 
capacity  required  for  the  primary  task.  It  is  expected  that  N190  might  be 
an  index  of  reserve  capacity  available  for  monitoring  events  outside  of  the 
primary  task  and  therefore  will  vary  inversely  with  the  independent  task 
difficulty  variable . Likewise , the  h'lOO  component  may  vary  in  a compara b la 
manner  but  to  a lesser  extent,  and  under  some  conditions , s".ch  as  I 1 
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easy  task,  the  P300  compono  : may  emerge  in  the  waveform . 

Additional  work  will  he  directed  at  single  trial  determinations  of 
reserve  capacity  based  upon  the  averaged  ERP  results  and  utilizing  the 
single  trial  waveforms  from  the  experiment . The  end  product  of  this 
first  experiment  will  then  bn  a calibration  procedure  for  using  the  ERP 
to  unobtrusively  monitor  subject  state. 

A complimentary  approach  will  be  used  to  evaluate  ERP  waveforms 
elicited  by  stimuli  which  are  directly  relevant  to  the  subject's  primary 
task.  On  the  basis  of  previous  work  it  can  be  predicted  that  certain 
components , namely  N100  and  P300,  are  enhanced  when  attention  is  paid  to 
particular  stimuli.  An  additional  component  labeled  the  "slow  wave"  (Sff) 
has  also  been  recently  discovered  and  seems  to  fall  within  the  class  of 
attention  related  potentials . That  such  enhancement  effects  occur  is 
well  documented , however,  no  systematic  attempts  have  been  made  to  deal 
with  the  effect  in  terms  of  quantifiable  measures  of  directed  attention . 

Such  measures  are  included  in  the  time  sharing  techniques  that  were 
developed  in  the  Aviation  Research  Labe catory  and  will  be  incorporated 
in  this  experiment.  The  experiment  will  follow  the  basic  paradigm  of 
the  secondary  task  technique;  however,  several  conditions  of  subtask 
priorities  will  be  investigated . Subtask  priorities  in  this  approach  are 
manipulated  by  changing  the  feedback  indicators . 

Two  types  of  data  are  relevant  to  this  approach,  the  stimuli  upon 
which  task  performance  is  contingent  and  stimuli  which  provide  feedback  to 
the  subject  as  to  his  performance.  Feedback  stimuli  suitable  for  ERP 
recording  can  be  incorporated  in  any  performance  task  including  the  track- 
ing task.  For  instance , tone  bursts  of  varying  frequency  can  be  presented 
whenever  the  subject  falls  below  certain  minimum  performance  criterion . By 
varying  the  .informative  value  of  feedback  stimuli  through  variations  in 
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redundancy,  rate  of  presentation,  and  clarity  of  the  stimuli  simultaneously 
in  several  channels  or  modalities  we  will  evaluate  the  usefulness  of  the 
P300  and  other  components  as  indices  of  focused  attention  and  information 
content  of  feedback  stimuli. 

Similarly  we  will  approach  the  problem  from  the  standpoint  of  the  task 
contingent  stimuli.  Presentations  of  target  conditions,  where  a target  may 
be  a brief  signal  or  change  in  a signal,  are  known  to  elicit  P300  components 
when  a target  is  detected  but  not  when  it  goes  undetected.  Also,  non-tar- 
gets which  are  similar  to  targets  may  also  elicit  P300s.  Thus  it  is  possi- 
ble to  probe  the  subject  within  the  set  of  task  relevant  stimuli  to  monitor 
his  degree  and  direction  of  focused  attention. 

The  data  sets  from  these  experiments  will  allow  us  to  compare  the  effi- 
ciencies of  assessing  subject  state  by  the  four  approaches,  monitoring  of 
the  irrelevant  probe  ERPs , relevant  feedback  ERPs , relevant  task  stimulus 
ERPs,  and  irrelevant  probe  stimulus  ERPs  within  a relevant  channel . 

2.  ERPs  in  the  CAI  Laboratory 

Though  the  application  of  ERP  techniques  to  a CAI  environment  is 
largely  an  unexplored  area  the  implementati.on  of  such  a program  is  expected 
to  yield  mutual  benefits . The  PLATO  lession  provides  an  environment  within 
which  highly  relevant , verbal  material  is  presented  to  a subject  who  pro- 
cessed the  information , makes  a decision , makes  a response  and  then  re- 
ceives feedback  regarding  that  response.  The  PLATO  " instructor " presently 
has  access  to  the  subject's  behavior  only  through  his  response  on  the  touch 
panel  or  keyboard.  Since  one  of  the  main  thrusts  of  ERP  research  has  been 
to  isolate  certain  ERP  correlates  of  information  processing  and  decision 
making  it  may  now  be  possible  to  supplement  the  instructor's  knowledge 
about  the  student  by  providing  measures  of  ERP  activity  during  the  question- 
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answer- feedback  sequence.  In  the  (distant)  future  it  may  be  possible  to 
eliminate  the  behavioral  response  altogether  and  roly  entirely  upon  ERP 
to  direct  the  course  of  a teaching  session. 

For  example,  in  addition  to  the  subject's  behavioral  response  the  in- 

<• 

structor-computer  can  monitor  the  ERP  elicited  bu  the  feedback  signal  on 
each  trial.  It  has  previously  been  shown  (by  members  of  this  laboratory) 
that  the  amplitude  and  latency  of  the  P300  component  following  a simple 
visual  feedback  signal  vary  as  a function  of  whether  that  signal  confirms 
that  the  response  was  correct  or  indicates  that  an  error  was  made.  Further- 
more , these  measures  vary  according  to  the  confidence  with  which  the  ori- 
ginal decision  was  made.  When  a subject  is  highly  certain  of  his  decision 
and  is  informed  that  the  answer  was  correct a small,  early  P300  component 
is  elicited.  However  when  told  that  an  equally  certain  answer  was  wrong 
the  ERP  shows  a much  larger  and  somewhat  later  P300  component.  When  the 
subject  is  uncertain  of  his  decision , P300  components  of  intermediate 
amplitude  and  latency  are  found.  Thus,  the  feedback  ERP  potentially  pro- 
vides a continuous  scale  along  which  the  subjects'  decisions  on  each 
trial  can  be  assessed  by  the  instructor  computer  to  determine  the  current 
level  of  understanding  within  the  lesson. 

The  above  results  apply  to  simple  feedback  signals,  the  flash  of  a 
colored  light  or  a tone  burst  of  a particular  frequency , in  psychophysical 
detection  and  discrimination  tasks.  We  must  determine  if  these  results 
will  transfer  to  a situation  wherein  linguistic  messages  are  presented 
visually  within  a realistic  learning  task. 

A simple  learning  task  will  bo  developed  or  taken  from  available  PLA TO 
lessons.  The  lesson  will  he  selected  so  that  it  presents  material  which 
is  novel  to  the  subject  population  and  yields  responses  which  span  a wide 
range  of  certainty  on  the  part  of  the  subject.  Confidence-rating  responses 
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will  be  collected  to  directly  assess  the  certainty  in  each  decision  and 
average  ERPs  will  be  collected  according  to  the  combination  of  confidence 
rating  of  the  decision  and  valence  of  the  feedback  (confirming  or  discon- 

firming)  . Additional  control  conditions  will  also  be  presented  within 

0 

which  non-in  formative  "feedback''  is  presented  and  waveforms  to  such 
stimuli  also  collected . 

This  initial  experiment  will  be  presented  visually  and  utilize  as  much 
of  the  standard  PLATO  protocol  as  possible  but  will  be  idealized  to  the 
extent  that  all  known  potential  artifacts  and  problems  that  may  occur  in 
a more  realistic  situation  are  eliminated . This  will  entail  fixed  record 
lengths , fixed  location  of  feedback  presentation,  fixed  inter-trial  inter- 
val, etc . Loosening  of  such  contraints  will  be  incorporated  in  subsequent 
experiments  when  the  effects  of  each  presentation  variable  arc  i worked  out. 

Data  analysis  will  consist  of  a detailed  analysis  of  the  ERP  waveforms 
which  is  standard  in  our  laboratory . This  includes  a factor  analysis  to 
isolate  components  of  the  complex  ERP  waveform.  The  resulting  components 
will  be  assessed  as  to  their  relationship  to  the  independent  variables 
(stimulus  "OK"  and  stimulus  "ERROR")  and  dependent  variables  (certainty 
and  stimulus  valence)  and  compared  with  analogous  components  from  feedback 
in  a simple  detection  task.  Of  special  interest  are  variations  in  com- 
ponent amplitude  over  the  surface  of  the  scalp  which  will  be  determined 
tiirough  the  use  of  multiple  (10-12)  channels  and  which  have  not  been 
adequately  measured  up  until  now.  Particular  emphasis  will  be  placed  on 
determining  which  aspects  of  the  ERP  waveform  are  most  powerful  in  segre- 
gating the  waveform  types  and  which  aspects  are  common  to  all  information 
processing  tasks.  Data  collected  here  will  be  incorporated  into  the  ex- 
panding data  base  we  have  from  a number  of  separate  tasks. 

Further  experiments  will  deal  in  greater  depth  with  other  aspects  of 
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t he  question-response- feedbac\  sequence . For  instance , it  is  expected  that 
the  pre- feedback  CWV  may  index  decision  ccrtai nty  and  feedback  expectancy 
in  a manner  complementary  to  the  feedback  ERP.  Likewise  there  is  an  ex- 
pected link  between  the  subject's  decision  and  the  time  of  occurrence  of 
his  behavioral  response . 

Other  variables  which  should  be  studied  are  the  information  content  of 
the  feedback  stimulus  (i.e.,  right-wrong  versus  the  correct  answer,  which 
then  must  be  interpreted  to  determine  the  valence,  and  vice-versa) , the 
response  feedback  delay,  spatial  position  of  the  feedback,  the  relative 
contrast  of  feedback  versus  the  array  already  on  the  screen,  and  many  other 
technical  and  procedural  points. 

It  is  expected  that  Experiment  I can  be  completed  within  the  bounds 
of  our  current  expertise,  given  that  the  required  equipment  is  available, 
and  should  lead  to  many  more  sophisticated  experiments  which  will  be  out- 
lined later. 

3.  Further  Developments  of  ERP  Vocabulary 

The  vocabulary  of  the  ERP  as  we  now  know  it  includes  a pre-signal  com- 
ponent (CNV) , a pre-activity  component  (readiness  potential) , two  signal 
related  components  (NlOO,  P200)  and  at  least  three  post-signal  components 
related  to  the  processing  of  significant  events  (N190 , P350,  and  SW) . While 
all  are  robust  phenomena  and  their  occurrence  can  often  be  predicted , it 
nan  be  safely  stated  that  neither  the  anatomical  nor  psychological  bases  of 
any  of  these  components  is  fully  understood.  In  addition,  the  improved 
resolving  power  of  the  techniques  of  ERP  recording  and  aiialysi s promise 
to  continue  providing  new  ERP  phenomena.  The  N190  and  SW  components , for 
instance , which  were  first  described  within  the  past  year,  went  unnoticed 
until  multi-channel  recording  was  used  and  subsequent  factor  analysis 
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techniques  applied  to  the  data  tp  isolate  the  otherwise  temporally  entan- 
gled components.  Thus  an  important  facet  of  this  work  effort  will  be  the 
continuation  of  study  programs  targeted  at  a complete  psychological 
descri ption  of  the  existing  vocabulary  and  identification  and  isolation 
of  new  ERP  co nponen ts . 

A description  of  the  N190  component  in  a performance  setting  and 
efficient  evaluation  of  it  as  a biocybernetic  tool  must  be  supported  by 
an  improved  understanding  of  the  basic  parameters  governing  its  behavior. 
Anecdotal  evidence  suggests  that  N190  is  more  pronounced  when  rare  low 
intensity  stimuli  are  introduced  into  a train  of  higher  intensity  tone 
bursts.  In  some  other  cases  it  appears  that  frequency  shifts  elicit 
clearer  N190s  than  do  intensity  shifts.  Clearly  it  is  important  to  deter- 
mine the  optimum  conditions  for  eliciting  N190 , thus  increasing  the  signal- 
to-noise  ratio,  if  it  is  to  be  used  efficiently  to  monitor  variations  in 
the  state  of  a performing  subject,  particularly  on  a single  trial  basis. 

To  do  tliis  a parametric  study  will  be  performed  wherein  stimulus  and  timing 
variables  such  as  frequency , intensity , direction , and  amount  of  intensity 
or  frequency  change  for  auditory  signals;  rate  or  stimulus  presentation 
and  density  of  rare  stimuli;  and  modality  of  presentation,  auditory , visual, 
ana  somestheiic. 

An  unexpected  result  from  a recent  study  conducted  in  our  laboratory 
is  that  the  N190  component  in  an  auditory  intensity  shift  experiment  is 
apparently  lateralized  to  the  right  hemisphere  and  largest  over  the  frontal 
cortex.  Post-hoc  explanations  of  such  a result  can  be  made  from  the 
literature  on  cerebral  dominance,  however  such  a finding  clearly  merits 
systematic  study  before  conclusions  are  reached.  Several  possibilities 
exist,  such  as  that  N190  may  be  modality  specific  and  distributed  differ- 
ently for,  say,  auditory  and  visual  stimuli;  it  nay  be  task-related  and 
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vary  according  to  the  design  of  the  primary  task,  for  instance,  manual 
versus  linguistic;  or  it  may  bo  a function  of  the  level  of  significance 
and  pattern  of  the  secondary  stimuli , as  in  a linguistic  versus  a musical 
pattern.  These  and  other  hypotheses  will  be  examined  in  conjunction  with 
experiments  conducted  in  the  Performance  and  CAI  studies. 

One  hypothesis  regarding  1 he  relationship  between  P300  and  cognitive 
events  is  that  P300  reflects  post-stimulus  processing  of  feedback  informa- 
tion provided  by  a signal  which  modifies  expectancies  of  receiving  subse- 
quent signals  or  modifies  subsequent  responses . As  such  the  P300  component 
will  depend  upon  the  quality  of  the  feedback  information  as  well  as  its 
usefulness  m determining  subsequent  behavior . In  a current  study  w2  have 
found  that  auditory  feedback  stimuli  indicating  the  success  or  failure  of 
preceding  motor  performance  elicit  different  amplitude  P300  components 
according  to  the  intensity  relationship  between  "correct"  and  "incorrect" 
stimuli.  Since  all  stimulus  pairings  are  readily  distinguishable  at  the 
level  of  a psychophysical  discrimination  and  are  presented  within  series 
which  are  identical  from.  the  aspect  of  motor  task  performance , an  additional 
factor  is  evidently  present  which  modifies  the  effectiveness  of  the  feed- 
back information  in  eliciting  P300.  We  plan  to  investigate  this  phenomenon 
by  first  determining  the  critical  parameters  of  stimuluation  which  cause 
such  an  effect,  such  as  variations  in  the  response- feedback  interval,  and 
then  determining  the  level  at  which  the  stimulus  information  is  modified, 
for  instance , does  the  discrimination  take  longer  for  some  stimulus  oairs 
even  though  the  final  decision  (as  measured  by  the  discriminability)  is 
the  same  for  all  pairs. 

In  another  ongoing  experiment  we  have  found  that  the  P300  component 
elicited  by  rare  target  signal  within  a train  of  non  targets  varies  widely 
in  amplitude  according  to  the  rate  at  whi ^ : stimuli  occur  and  according  to 
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the  randomness  of  the  inter-stimulus  interval.  Prior  to  establishing  any 
general  predictive  hypothesis  regarding  P300 , th j relative  importance  of 
each  of  these  experimental  variables  and  any  possible  interactions  must  be 
determined.  We  plan  to  conduct  a thorough  parametric  study  of  this  effect. 

Prior  to  expected  signals  which  initiate  motor  operations  and  cognitive 
events,  two  ERP  waves  have  been  described,  the  readiness  potential  and  the 
CNV.  The  relationship  between  the  two  ERP  components  and  the  relevant 

i 

behavioral  variables  are  as  yet  not  clear.  In  fact  it  has  not  been  es- 
tablished whether  they  represent  differing  brain  processes , one  unique  to 
motor  tasks  and  one  to  cognitive  events  or  whether  the  components  reflect 
a general  preparatory  process  common  to  both  types  of  task  but  localized  to 
the  particular  areas  of  the  brain  involved  most  heavily  in  each  task.  It 
will  truly  enhance  the  understanding  of  brain-behavior  relationships  if  we 
could  determine  which  brain  regions  are  involved  in  given  behavioral  tasks 
by  assessing  the  degree  to  which  they  arc  "readied"  in  anticipation  of  pro- 
cessing. Our  approach  to  the  task  will  be  to  map  the  scalp  distributions 
of  CNVs  when  subjects  engage  in  different  behaviors  designed  to  shift  the  locus 
of  major  areas  of  brain  involvement.  Careful  analysis  of  asymmetric  scalp  dis- 
tributions and  time  courses  of  the  CNV  across  and  within  tasks  should  aid  in 
the  determination  of  the  basis  of  the  CNV. 

A question  that  merits  careful  consideration  is  that  of  the  relation  be- 
tween CNV-like  potentials  and  Reaction  Time.  The  low  correlations  between 
CNV  amplitude  and  reaction  time  when  measured  on  a trial  by  trial  basis 
are  quite  difficult  to  interpret  within  the  context  of  accepted  accounts 
of  the  CNV.  If  the  CNV  represents  a cortical  attempt  to  optimize  performance 
then  one  would  expect  a correlation  between  optimal  performance  and  the  CNV. 

There  arr  at  least  two  possible  explanations  of  the  low  correlations 
reported  in  the  literature , each  leading  to  an  important  series  of  experiments . 
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First  wo  note  tha t the  Reaction  Time  is  a multiply  determined  interval 
representing  the  total  duration  of  serial,  and  parallel,  stages  in  the 
stimulus  recaption-response  execution  sequence.  Among  such  stages  are 
stimulus  reception , stimulus  recognition , response  selection , and  re- 
sponse execution . The  CNV  may  well  reflect  proctisses  germane  to  one  such 
stage  and  not  to  the  others.  We  plan  a series  of  studies  in  which  Reaction 
Time  variance  would  be  partitioned  into  its  constituents  and  each  consti- 
tuent will  be  separately  related  to  the  CNV.  This  will,  of  course,  provide 
a finely  tuned  analysis  of  the  behavioral  significance  of  the  CNV . 

An  alternate  explanation  for  the  weakness  of  the  CNV/RT  relationship 
is  that  CNV  amplitude  varies  hetween  trials  as  a function  of,  say,  sub- 
ject's reserve  capacity,  in  order  to  maintain  a constant,  optimal,  RT . If 
this  is  the  case  the  correlation  between  CNV  and  RT  will  indeed  be  small, 
even  though  there  is  a strong  causal  relationship  between  the  CNV  and  RT. 

This  hypothesis  can  be  tested  in  a setting  which  permits  measures  of 
"reserve  capacity " while  the  CNV  and  RT  are  determined.  This  setting  will 
be  provided  in  the  laboratory  setting  we  are  developing . 

The  recently  described  SW  component  is,  because  of  its  slow  time  course, 
possibly  related  to  other  slow  potential  changes  such  as  the  CNV.  In  some 
paradigms  the  presignal  CNV  can  he  seen  to  "resolve" , or  return  to  an  equi- 
potential  condition,  following  a stimulus  presentation , causing  what  appears 
to  be  a positive  wave  if  only  post-signal  events  are  analyzed . The  SW  may 
be  such  a phenomenon.  If  so,  it  would  be  the  offset  of  a very  unique  CNV 
because  of  its  inferred  distribution,  which  would  differ  from  previously 
described  CNVs,  and,  because  no  phasic  CNVs  occur  between  stimuli  the 
phenomenon  would  have  to  be  a tonic  CNV,  something  that:  has  not  yet  been 
reported.  We  are  planning  to  investigate  this  question  using  multi-channel 
D.C.  recordings  and  a procedure  to  pick  up  both  the  RW  component  and  the 


onset  of  any  tonic  CNV. 


